Multi-Class Image Classification Model for Food 101 Identification Using TensorFlow Take 1¶

David Lowe¶

July 11, 2022¶

SUMMARY: This project aims to construct a predictive model using a TensorFlow convolutional neural network (CNN) and document the end-to-end steps using a template. The Belgium Traffic Sign dataset is a multi-class classification situation where we attempt to predict one of several (more than two) possible outcomes.

INTRODUCTION: This dataset includes 101 food categories, totaling 101,000 images. The research team provided 250 manually reviewed test images and 750 training images for each food type. All images were rescaled to have a maximum side length of 512 pixels.

ANALYSIS: The ResNet50V2 model's performance achieved an accuracy score of 81.19% after ten epochs using the training dataset. When we applied the model to the validation dataset, the model achieved an accuracy score of 61.24%.

CONCLUSION: In this iteration, the TensorFlow ResNet50V2 CNN model appeared suitable for modeling this dataset.

Dataset ML Model: Multi-Class classification with numerical features

Dataset Used: Food-101 – Mining Discriminative Components with Random Forests

Dataset Reference: https://data.vision.ee.ethz.ch/cvl/datasets_extra/food-101/

One source of potential performance benchmarks: https://data.vision.ee.ethz.ch/cvl/datasets_extra/food-101/

Task 1 - Prepare Environment¶

In [1]:
# Retrieve CPU information from the system
ncpu = !nproc
print("The number of available CPUs is:", ncpu[0])

The number of available CPUs is: 2
In [2]:
# Retrieve memory configuration information
from psutil import virtual_memory
ram_gb = virtual_memory().total / 1e9
print('Your runtime has {:.1f} gigabytes of available RAM\n'.format(ram_gb))

Your runtime has 13.6 gigabytes of available RAM
In [3]:
# Retrieve GPU configuration information
gpu_info = !nvidia-smi
gpu_info = '\n'.join(gpu_info)
print(gpu_info)

Sun Jul 10 20:41:09 2022       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 460.32.03    Driver Version: 460.32.03    CUDA Version: 11.2     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  Tesla V100-SXM2...  Off  | 00000000:00:04.0 Off |                    0 |
| N/A   43C    P0    24W / 300W |      0MiB / 16160MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

1.a) Load libraries and modules¶

In [4]:
# Set the random seed number for reproducible results
RNG_SEED = 888
In [5]:
import random
random.seed(RNG_SEED)
import numpy as np
np.random.seed(RNG_SEED)
import pandas as pd
import matplotlib.pyplot as plt
import matplotlib.image as mpimg
import os
import sys
import math
# import boto3
import zipfile
from datetime import datetime
from sklearn.metrics import classification_report
from sklearn.metrics import confusion_matrix
from sklearn.metrics import accuracy_score

import tensorflow as tf
tf.random.set_seed(RNG_SEED)
from tensorflow import keras
from tensorflow.keras.callbacks import ReduceLROnPlateau
from tensorflow.keras.preprocessing.image import ImageDataGenerator

1.b) Set up the controlling parameters and functions¶

In [6]:
# Begin the timer for the script processing
START_TIME_SCRIPT = datetime.now()
In [7]:
# Set up the number of CPU cores available for multi-thread processing
N_JOBS = 1

# Set up the flag to stop sending progress emails (setting to True will send status emails!)
NOTIFY_STATUS = False

# Set the percentage sizes for splitting the dataset
TEST_SET_RATIO = 0.1
VAL_SET_RATIO = 0.1

# Set the number of folds for cross validation
N_FOLDS = 5
N_ITERATIONS = 1

# Set various default modeling parameters
DEFAULT_LOSS = 'categorical_crossentropy'
DEFAULT_METRICS = ['accuracy']
DEFAULT_OPTIMIZER = tf.keras.optimizers.Adam(learning_rate=0.0001)
CLASSIFIER_ACTIVATION = 'softmax'
MAX_EPOCHS = 10
BATCH_SIZE = 32
NUM_CLASSES = 101
# CLASS_LABELS = []
# CLASS_NAMES = []
# RAW_IMAGE_SIZE = (250, 250)
TARGET_IMAGE_SIZE = (224, 224)
INPUT_IMAGE_SHAPE = (TARGET_IMAGE_SIZE[0], TARGET_IMAGE_SIZE[1], 3)

# Define the labels to use for graphing the data
TRAIN_METRIC = "accuracy"
VALIDATION_METRIC = "val_accuracy"
TRAIN_LOSS = "loss"
VALIDATION_LOSS = "val_loss"

# Define the directory locations and file names
STAGING_DIR = 'staging/'
TRAIN_DIR = 'food-101/images/'
# VALID_DIR = ''
# TEST_DIR = ''
TRAIN_DATASET = 'food-101.tar.gz'
# VALID_DATASET = ''
# TEST_DATASET = ''
TRAIN_LABELS = 'food-101/meta/train.txt'
# VALID_LABELS = ''
TEST_LABELS = 'food-101/meta/test.txt'
# OUTPUT_DIR = 'staging/'
# SAMPLE_SUBMISSION_CSV = 'sample_submission.csv'
# FINAL_SUBMISSION_CSV = 'submission.csv'

# Check the number of GPUs accessible through TensorFlow
print('Num GPUs Available:', len(tf.config.list_physical_devices('GPU')))

# Print out the TensorFlow version for confirmation
print('TensorFlow version:', tf.__version__)

Num GPUs Available: 1
TensorFlow version: 2.8.2
In [8]:
# Set up the email notification function
def status_notify(msg_text):
    access_key = os.environ.get('SNS_ACCESS_KEY')
    secret_key = os.environ.get('SNS_SECRET_KEY')
    aws_region = os.environ.get('SNS_AWS_REGION')
    topic_arn = os.environ.get('SNS_TOPIC_ARN')
    if (access_key is None) or (secret_key is None) or (aws_region is None):
        sys.exit("Incomplete notification setup info. Script Processing Aborted!!!")
    sns = boto3.client('sns', aws_access_key_id=access_key, aws_secret_access_key=secret_key, region_name=aws_region)
    response = sns.publish(TopicArn=topic_arn, Message=msg_text)
    if response['ResponseMetadata']['HTTPStatusCode'] != 200 :
        print('Status notification not OK with HTTP status code:', response['ResponseMetadata']['HTTPStatusCode'])
In [9]:
if NOTIFY_STATUS: status_notify('(TensorFlow Multi-Class) Task 1 - Prepare Environment completed on ' + datetime.now().strftime('%A %B %d, %Y %I:%M:%S %p'))

Task 2 - Load and Prepare Images¶

In [10]:
if NOTIFY_STATUS: status_notify('(TensorFlow Multi-Class) Task 2 - Load and Prepare Images has begun on ' + datetime.now().strftime('%A %B %d, %Y %I:%M:%S %p'))
In [11]:
# Clean up the old files and download directories before receiving new ones
!rm -rf food-101/

if not os.path.exists(TRAIN_DATASET):
    !wget https://dainesanalytics.com/datasets/tensorflow-food-101/food-101.tar.gz
!tar -xzf food-101.tar.gz
!du -h

60K	./.config/logs/2022.07.06
64K	./.config/logs
8.0K	./.config/configurations
96K	./.config
4.1M	./food-101/meta
48M	./food-101/images/foie_gras
47M	./food-101/images/samosa
56M	./food-101/images/baby_back_ribs
53M	./food-101/images/ramen
50M	./food-101/images/beef_tartare
49M	./food-101/images/breakfast_burrito
48M	./food-101/images/scallops
51M	./food-101/images/ceviche
55M	./food-101/images/fried_rice
52M	./food-101/images/lasagna
53M	./food-101/images/donuts
49M	./food-101/images/cup_cakes
49M	./food-101/images/french_onion_soup
43M	./food-101/images/panna_cotta
52M	./food-101/images/falafel
56M	./food-101/images/french_toast
57M	./food-101/images/takoyaki
55M	./food-101/images/tacos
50M	./food-101/images/sashimi
42M	./food-101/images/miso_soup
45M	./food-101/images/spring_rolls
49M	./food-101/images/macarons
52M	./food-101/images/chicken_curry
47M	./food-101/images/tuna_tartare
53M	./food-101/images/prime_rib
42M	./food-101/images/frozen_yogurt
52M	./food-101/images/grilled_cheese_sandwich
48M	./food-101/images/hot_and_sour_soup
46M	./food-101/images/chocolate_mousse
49M	./food-101/images/apple_pie
56M	./food-101/images/bibimbap
47M	./food-101/images/cheese_plate
50M	./food-101/images/french_fries
54M	./food-101/images/lobster_roll_sandwich
48M	./food-101/images/risotto
56M	./food-101/images/beef_carpaccio
46M	./food-101/images/eggs_benedict
42M	./food-101/images/lobster_bisque
48M	./food-101/images/creme_brulee
52M	./food-101/images/huevos_rancheros
49M	./food-101/images/baklava
51M	./food-101/images/spaghetti_carbonara
53M	./food-101/images/chicken_wings
49M	./food-101/images/hummus
50M	./food-101/images/edamame
52M	./food-101/images/grilled_salmon
48M	./food-101/images/ravioli
49M	./food-101/images/carrot_cake
50M	./food-101/images/shrimp_and_grits
48M	./food-101/images/ice_cream
50M	./food-101/images/red_velvet_cake
51M	./food-101/images/caprese_salad
50M	./food-101/images/sushi
55M	./food-101/images/spaghetti_bolognese
53M	./food-101/images/bruschetta
56M	./food-101/images/greek_salad
54M	./food-101/images/escargots
53M	./food-101/images/beet_salad
48M	./food-101/images/deviled_eggs
47M	./food-101/images/dumplings
56M	./food-101/images/seaweed_salad
53M	./food-101/images/omelette
54M	./food-101/images/mussels
50M	./food-101/images/crab_cakes
48M	./food-101/images/macaroni_and_cheese
50M	./food-101/images/chocolate_cake
48M	./food-101/images/filet_mignon
49M	./food-101/images/croque_madame
53M	./food-101/images/beignets
54M	./food-101/images/guacamole
53M	./food-101/images/pulled_pork_sandwich
54M	./food-101/images/nachos
58M	./food-101/images/pizza
51M	./food-101/images/chicken_quesadilla
52M	./food-101/images/pork_chop
53M	./food-101/images/steak
52M	./food-101/images/pancakes
50M	./food-101/images/gnocchi
52M	./food-101/images/pho
52M	./food-101/images/waffles
50M	./food-101/images/onion_rings
49M	./food-101/images/hot_dog
50M	./food-101/images/tiramisu
50M	./food-101/images/gyoza
46M	./food-101/images/strawberry_shortcake
53M	./food-101/images/club_sandwich
47M	./food-101/images/bread_pudding
55M	./food-101/images/caesar_salad
51M	./food-101/images/hamburger
53M	./food-101/images/poutine
56M	./food-101/images/pad_thai
55M	./food-101/images/oysters
52M	./food-101/images/fried_calamari
50M	./food-101/images/churros
52M	./food-101/images/fish_and_chips
52M	./food-101/images/garlic_bread
43M	./food-101/images/clam_chowder
59M	./food-101/images/paella
52M	./food-101/images/peking_duck
51M	./food-101/images/cannoli
44M	./food-101/images/cheesecake
5.0G	./food-101/images
5.0G	./food-101
55M	./sample_data
9.7G	.
In [12]:
CLASS_LABELS = os.listdir(TRAIN_DIR)
print(CLASS_LABELS)

['foie_gras', 'samosa', 'baby_back_ribs', 'ramen', 'beef_tartare', 'breakfast_burrito', 'scallops', 'ceviche', 'fried_rice', 'lasagna', 'donuts', 'cup_cakes', 'french_onion_soup', 'panna_cotta', 'falafel', 'french_toast', 'takoyaki', 'tacos', 'sashimi', 'miso_soup', 'spring_rolls', 'macarons', 'chicken_curry', 'tuna_tartare', 'prime_rib', 'frozen_yogurt', 'grilled_cheese_sandwich', 'hot_and_sour_soup', 'chocolate_mousse', 'apple_pie', 'bibimbap', 'cheese_plate', 'french_fries', 'lobster_roll_sandwich', 'risotto', 'beef_carpaccio', 'eggs_benedict', 'lobster_bisque', 'creme_brulee', 'huevos_rancheros', 'baklava', 'spaghetti_carbonara', 'chicken_wings', 'hummus', 'edamame', 'grilled_salmon', 'ravioli', 'carrot_cake', 'shrimp_and_grits', 'ice_cream', 'red_velvet_cake', 'caprese_salad', 'sushi', 'spaghetti_bolognese', 'bruschetta', 'greek_salad', 'escargots', 'beet_salad', 'deviled_eggs', 'dumplings', 'seaweed_salad', 'omelette', 'mussels', 'crab_cakes', 'macaroni_and_cheese', 'chocolate_cake', 'filet_mignon', 'croque_madame', 'beignets', 'guacamole', 'pulled_pork_sandwich', 'nachos', 'pizza', 'chicken_quesadilla', 'pork_chop', 'steak', 'pancakes', 'gnocchi', 'pho', 'waffles', 'onion_rings', 'hot_dog', 'tiramisu', 'gyoza', 'strawberry_shortcake', 'club_sandwich', 'bread_pudding', 'caesar_salad', 'hamburger', 'poutine', 'pad_thai', 'oysters', 'fried_calamari', 'churros', 'fish_and_chips', 'garlic_bread', 'clam_chowder', 'paella', 'peking_duck', 'cannoli', 'cheesecake']
In [13]:
# Brief listing of training image files for each class
for c_label in CLASS_LABELS:
    training_class_dir = os.path.join(TRAIN_DIR, c_label)
    training_class_files = os.listdir(training_class_dir)
    print('Number of training images for', c_label, ':', len(os.listdir(training_class_dir)))
    print('Training samples for', c_label, ':', training_class_files[:5],'\n')

Number of training images for foie_gras : 1000
Training samples for foie_gras : ['982330.jpg', '1402073.jpg', '651184.jpg', '1241360.jpg', '1675850.jpg'] 

Number of training images for samosa : 1000
Training samples for samosa : ['1016296.jpg', '2239746.jpg', '2095432.jpg', '2636065.jpg', '1750149.jpg'] 

Number of training images for baby_back_ribs : 1000
Training samples for baby_back_ribs : ['41235.jpg', '652170.jpg', '299268.jpg', '2529527.jpg', '2820528.jpg'] 

Number of training images for ramen : 1000
Training samples for ramen : ['482695.jpg', '726528.jpg', '456047.jpg', '27878.jpg', '3799319.jpg'] 

Number of training images for beef_tartare : 1000
Training samples for beef_tartare : ['1991843.jpg', '2557204.jpg', '609036.jpg', '2851803.jpg', '2280171.jpg'] 

Number of training images for breakfast_burrito : 1000
Training samples for breakfast_burrito : ['2601355.jpg', '3161330.jpg', '3427294.jpg', '998295.jpg', '2990962.jpg'] 

Number of training images for scallops : 1000
Training samples for scallops : ['100757.jpg', '726886.jpg', '2449172.jpg', '552120.jpg', '367585.jpg'] 

Number of training images for ceviche : 1000
Training samples for ceviche : ['2592271.jpg', '398695.jpg', '2020581.jpg', '256358.jpg', '309718.jpg'] 

Number of training images for fried_rice : 1000
Training samples for fried_rice : ['805390.jpg', '2986320.jpg', '167726.jpg', '108226.jpg', '2390079.jpg'] 

Number of training images for lasagna : 1000
Training samples for lasagna : ['3874120.jpg', '3474501.jpg', '3181564.jpg', '398728.jpg', '3798089.jpg'] 

Number of training images for donuts : 1000
Training samples for donuts : ['861022.jpg', '1878848.jpg', '1479615.jpg', '98500.jpg', '1218134.jpg'] 

Number of training images for cup_cakes : 1000
Training samples for cup_cakes : ['1191407.jpg', '275780.jpg', '3159087.jpg', '3832250.jpg', '2538109.jpg'] 

Number of training images for french_onion_soup : 1000
Training samples for french_onion_soup : ['397460.jpg', '1617170.jpg', '605550.jpg', '3479388.jpg', '339850.jpg'] 

Number of training images for panna_cotta : 1000
Training samples for panna_cotta : ['2778700.jpg', '2469457.jpg', '1652688.jpg', '3865077.jpg', '2301322.jpg'] 

Number of training images for falafel : 1000
Training samples for falafel : ['1184616.jpg', '1611924.jpg', '2441329.jpg', '2566338.jpg', '843221.jpg'] 

Number of training images for french_toast : 1000
Training samples for french_toast : ['2771213.jpg', '1324607.jpg', '1382841.jpg', '3493695.jpg', '1285422.jpg'] 

Number of training images for takoyaki : 1000
Training samples for takoyaki : ['455782.jpg', '262389.jpg', '908475.jpg', '2889718.jpg', '837193.jpg'] 

Number of training images for tacos : 1000
Training samples for tacos : ['3806565.jpg', '1980126.jpg', '2315447.jpg', '1316537.jpg', '3377716.jpg'] 

Number of training images for sashimi : 1000
Training samples for sashimi : ['305161.jpg', '6194.jpg', '311755.jpg', '1503204.jpg', '3621134.jpg'] 

Number of training images for miso_soup : 1000
Training samples for miso_soup : ['229663.jpg', '107031.jpg', '580490.jpg', '1307603.jpg', '1227861.jpg'] 

Number of training images for spring_rolls : 1000
Training samples for spring_rolls : ['3856737.jpg', '3661288.jpg', '1399265.jpg', '691378.jpg', '2671128.jpg'] 

Number of training images for macarons : 1000
Training samples for macarons : ['3062770.jpg', '457821.jpg', '886478.jpg', '946136.jpg', '2001438.jpg'] 

Number of training images for chicken_curry : 1000
Training samples for chicken_curry : ['26605.jpg', '1906787.jpg', '962899.jpg', '3336740.jpg', '3679727.jpg'] 

Number of training images for tuna_tartare : 1000
Training samples for tuna_tartare : ['145826.jpg', '2813744.jpg', '34487.jpg', '3561619.jpg', '1645789.jpg'] 

Number of training images for prime_rib : 1000
Training samples for prime_rib : ['3034286.jpg', '53393.jpg', '1255939.jpg', '3028756.jpg', '3254765.jpg'] 

Number of training images for frozen_yogurt : 1000
Training samples for frozen_yogurt : ['2418059.jpg', '1596313.jpg', '1019290.jpg', '3268279.jpg', '832227.jpg'] 

Number of training images for grilled_cheese_sandwich : 1000
Training samples for grilled_cheese_sandwich : ['679389.jpg', '2235066.jpg', '314341.jpg', '1854869.jpg', '133378.jpg'] 

Number of training images for hot_and_sour_soup : 1000
Training samples for hot_and_sour_soup : ['943151.jpg', '230642.jpg', '2909515.jpg', '325791.jpg', '2364496.jpg'] 

Number of training images for chocolate_mousse : 1000
Training samples for chocolate_mousse : ['1206320.jpg', '1377749.jpg', '45918.jpg', '1688182.jpg', '402863.jpg'] 

Number of training images for apple_pie : 1000
Training samples for apple_pie : ['238868.jpg', '3831869.jpg', '1005649.jpg', '127721.jpg', '1547651.jpg'] 

Number of training images for bibimbap : 1000
Training samples for bibimbap : ['3562104.jpg', '607312.jpg', '338795.jpg', '70746.jpg', '1373847.jpg'] 

Number of training images for cheese_plate : 1000
Training samples for cheese_plate : ['1667697.jpg', '3697495.jpg', '2705601.jpg', '3305631.jpg', '286905.jpg'] 

Number of training images for french_fries : 1000
Training samples for french_fries : ['3893313.jpg', '3002687.jpg', '2881817.jpg', '57594.jpg', '619413.jpg'] 

Number of training images for lobster_roll_sandwich : 1000
Training samples for lobster_roll_sandwich : ['3432474.jpg', '2370233.jpg', '1956592.jpg', '1970644.jpg', '140205.jpg'] 

Number of training images for risotto : 1000
Training samples for risotto : ['2174898.jpg', '2393988.jpg', '2014866.jpg', '3836575.jpg', '1243941.jpg'] 

Number of training images for beef_carpaccio : 1000
Training samples for beef_carpaccio : ['821784.jpg', '1811744.jpg', '491638.jpg', '1314036.jpg', '127274.jpg'] 

Number of training images for eggs_benedict : 1000
Training samples for eggs_benedict : ['2360670.jpg', '3803684.jpg', '2336488.jpg', '3603645.jpg', '3022255.jpg'] 

Number of training images for lobster_bisque : 1000
Training samples for lobster_bisque : ['1964693.jpg', '2108010.jpg', '3489265.jpg', '3437858.jpg', '2007600.jpg'] 

Number of training images for creme_brulee : 1000
Training samples for creme_brulee : ['911836.jpg', '1365780.jpg', '39615.jpg', '1184409.jpg', '2585208.jpg'] 

Number of training images for huevos_rancheros : 1000
Training samples for huevos_rancheros : ['537415.jpg', '2167700.jpg', '1259536.jpg', '1473184.jpg', '1210215.jpg'] 

Number of training images for baklava : 1000
Training samples for baklava : ['2546142.jpg', '173868.jpg', '3391092.jpg', '3400877.jpg', '1491090.jpg'] 

Number of training images for spaghetti_carbonara : 1000
Training samples for spaghetti_carbonara : ['1566710.jpg', '3068667.jpg', '2240281.jpg', '573828.jpg', '1913602.jpg'] 

Number of training images for chicken_wings : 1000
Training samples for chicken_wings : ['3473903.jpg', '997781.jpg', '1617605.jpg', '2726284.jpg', '275243.jpg'] 

Number of training images for hummus : 1000
Training samples for hummus : ['2962849.jpg', '2801564.jpg', '328911.jpg', '3770127.jpg', '3004527.jpg'] 

Number of training images for edamame : 1000
Training samples for edamame : ['2057674.jpg', '3260278.jpg', '3560910.jpg', '1426533.jpg', '3522373.jpg'] 

Number of training images for grilled_salmon : 1000
Training samples for grilled_salmon : ['990126.jpg', '1427596.jpg', '1921126.jpg', '559544.jpg', '1665568.jpg'] 

Number of training images for ravioli : 1000
Training samples for ravioli : ['945322.jpg', '2454419.jpg', '1088753.jpg', '2537524.jpg', '2877580.jpg'] 

Number of training images for carrot_cake : 1000
Training samples for carrot_cake : ['48061.jpg', '3126293.jpg', '962652.jpg', '2476666.jpg', '3411379.jpg'] 

Number of training images for shrimp_and_grits : 1000
Training samples for shrimp_and_grits : ['1400608.jpg', '1869163.jpg', '1116552.jpg', '538291.jpg', '289205.jpg'] 

Number of training images for ice_cream : 1000
Training samples for ice_cream : ['90167.jpg', '1793992.jpg', '2662236.jpg', '3845977.jpg', '1508158.jpg'] 

Number of training images for red_velvet_cake : 1000
Training samples for red_velvet_cake : ['356297.jpg', '3231500.jpg', '1445335.jpg', '3202454.jpg', '2186579.jpg'] 

Number of training images for caprese_salad : 1000
Training samples for caprese_salad : ['140286.jpg', '738701.jpg', '1247706.jpg', '2082783.jpg', '2791136.jpg'] 

Number of training images for sushi : 1000
Training samples for sushi : ['215265.jpg', '45515.jpg', '2049021.jpg', '3886015.jpg', '1591167.jpg'] 

Number of training images for spaghetti_bolognese : 1000
Training samples for spaghetti_bolognese : ['2185402.jpg', '1277517.jpg', '2541336.jpg', '2143809.jpg', '3203671.jpg'] 

Number of training images for bruschetta : 1000
Training samples for bruschetta : ['745164.jpg', '754320.jpg', '1908995.jpg', '463566.jpg', '1188689.jpg'] 

Number of training images for greek_salad : 1000
Training samples for greek_salad : ['1101689.jpg', '1801087.jpg', '1010520.jpg', '851863.jpg', '1912440.jpg'] 

Number of training images for escargots : 1000
Training samples for escargots : ['3434848.jpg', '1046909.jpg', '1965742.jpg', '2041923.jpg', '1763370.jpg'] 

Number of training images for beet_salad : 1000
Training samples for beet_salad : ['3611628.jpg', '148028.jpg', '1288727.jpg', '3022735.jpg', '2233387.jpg'] 

Number of training images for deviled_eggs : 1000
Training samples for deviled_eggs : ['2610611.jpg', '2029901.jpg', '1296591.jpg', '1831938.jpg', '1276492.jpg'] 

Number of training images for dumplings : 1000
Training samples for dumplings : ['1690805.jpg', '786848.jpg', '521158.jpg', '3152671.jpg', '310707.jpg'] 

Number of training images for seaweed_salad : 1000
Training samples for seaweed_salad : ['1141081.jpg', '2135717.jpg', '3404998.jpg', '1683342.jpg', '1377242.jpg'] 

Number of training images for omelette : 1000
Training samples for omelette : ['386378.jpg', '1534643.jpg', '507502.jpg', '921299.jpg', '3520398.jpg'] 

Number of training images for mussels : 1000
Training samples for mussels : ['1242929.jpg', '3136905.jpg', '313848.jpg', '3568376.jpg', '3642419.jpg'] 

Number of training images for crab_cakes : 1000
Training samples for crab_cakes : ['2221803.jpg', '2594773.jpg', '1789347.jpg', '3030758.jpg', '889544.jpg'] 

Number of training images for macaroni_and_cheese : 1000
Training samples for macaroni_and_cheese : ['744203.jpg', '1372196.jpg', '371248.jpg', '379982.jpg', '3160231.jpg'] 

Number of training images for chocolate_cake : 1000
Training samples for chocolate_cake : ['3047338.jpg', '3763921.jpg', '2581555.jpg', '1061106.jpg', '34516.jpg'] 

Number of training images for filet_mignon : 1000
Training samples for filet_mignon : ['2150839.jpg', '2100813.jpg', '374980.jpg', '2624923.jpg', '54582.jpg'] 

Number of training images for croque_madame : 1000
Training samples for croque_madame : ['1763826.jpg', '2891420.jpg', '1009303.jpg', '1498787.jpg', '452256.jpg'] 

Number of training images for beignets : 1000
Training samples for beignets : ['1619538.jpg', '978771.jpg', '1545060.jpg', '1648484.jpg', '2737988.jpg'] 

Number of training images for guacamole : 1000
Training samples for guacamole : ['160301.jpg', '1686131.jpg', '792293.jpg', '2629299.jpg', '2276850.jpg'] 

Number of training images for pulled_pork_sandwich : 1000
Training samples for pulled_pork_sandwich : ['2941105.jpg', '586693.jpg', '239894.jpg', '1762035.jpg', '1660736.jpg'] 

Number of training images for nachos : 1000
Training samples for nachos : ['3639989.jpg', '3804850.jpg', '135365.jpg', '1014846.jpg', '1986110.jpg'] 

Number of training images for pizza : 1000
Training samples for pizza : ['868789.jpg', '274945.jpg', '3581181.jpg', '369017.jpg', '1038357.jpg'] 

Number of training images for chicken_quesadilla : 1000
Training samples for chicken_quesadilla : ['2354586.jpg', '1704679.jpg', '375874.jpg', '993661.jpg', '447916.jpg'] 

Number of training images for pork_chop : 1000
Training samples for pork_chop : ['2126374.jpg', '3167838.jpg', '1050283.jpg', '1437232.jpg', '2244599.jpg'] 

Number of training images for steak : 1000
Training samples for steak : ['215222.jpg', '1094883.jpg', '1264154.jpg', '2331076.jpg', '3271253.jpg'] 

Number of training images for pancakes : 1000
Training samples for pancakes : ['96818.jpg', '1472280.jpg', '2256561.jpg', '3906138.jpg', '232916.jpg'] 

Number of training images for gnocchi : 1000
Training samples for gnocchi : ['2827633.jpg', '66896.jpg', '3841328.jpg', '1996096.jpg', '56717.jpg'] 

Number of training images for pho : 1000
Training samples for pho : ['2381496.jpg', '1275024.jpg', '3598157.jpg', '745846.jpg', '3016296.jpg'] 

Number of training images for waffles : 1000
Training samples for waffles : ['885452.jpg', '3491832.jpg', '1350003.jpg', '3402541.jpg', '602347.jpg'] 

Number of training images for onion_rings : 1000
Training samples for onion_rings : ['2030344.jpg', '3246848.jpg', '294279.jpg', '2823093.jpg', '3869178.jpg'] 

Number of training images for hot_dog : 1000
Training samples for hot_dog : ['127252.jpg', '2699758.jpg', '2779211.jpg', '607287.jpg', '2339412.jpg'] 

Number of training images for tiramisu : 1000
Training samples for tiramisu : ['2662861.jpg', '2241680.jpg', '1798786.jpg', '3776018.jpg', '2786037.jpg'] 

Number of training images for gyoza : 1000
Training samples for gyoza : ['2759564.jpg', '3110435.jpg', '2884263.jpg', '214110.jpg', '52405.jpg'] 

Number of training images for strawberry_shortcake : 1000
Training samples for strawberry_shortcake : ['1781668.jpg', '626965.jpg', '3125727.jpg', '325062.jpg', '2537353.jpg'] 

Number of training images for club_sandwich : 1000
Training samples for club_sandwich : ['120348.jpg', '1418469.jpg', '1107404.jpg', '1104022.jpg', '415755.jpg'] 

Number of training images for bread_pudding : 1000
Training samples for bread_pudding : ['483295.jpg', '1352570.jpg', '100939.jpg', '1635578.jpg', '2453557.jpg'] 

Number of training images for caesar_salad : 1000
Training samples for caesar_salad : ['1480826.jpg', '1262561.jpg', '2243999.jpg', '294440.jpg', '3671839.jpg'] 

Number of training images for hamburger : 1000
Training samples for hamburger : ['1286359.jpg', '102005.jpg', '2100953.jpg', '973733.jpg', '1210060.jpg'] 

Number of training images for poutine : 1000
Training samples for poutine : ['1040500.jpg', '459088.jpg', '76915.jpg', '3576310.jpg', '2104060.jpg'] 

Number of training images for pad_thai : 1000
Training samples for pad_thai : ['102781.jpg', '2865665.jpg', '3775499.jpg', '1419147.jpg', '761235.jpg'] 

Number of training images for oysters : 1000
Training samples for oysters : ['3740978.jpg', '1719514.jpg', '2546981.jpg', '3618467.jpg', '333702.jpg'] 

Number of training images for fried_calamari : 1000
Training samples for fried_calamari : ['2243009.jpg', '3319672.jpg', '3667448.jpg', '3056594.jpg', '1549831.jpg'] 

Number of training images for churros : 1000
Training samples for churros : ['2227644.jpg', '671479.jpg', '199032.jpg', '1642518.jpg', '3305528.jpg'] 

Number of training images for fish_and_chips : 1000
Training samples for fish_and_chips : ['2531288.jpg', '528885.jpg', '3222017.jpg', '1225146.jpg', '3466493.jpg'] 

Number of training images for garlic_bread : 1000
Training samples for garlic_bread : ['3314493.jpg', '2025171.jpg', '2366907.jpg', '3039541.jpg', '2587195.jpg'] 

Number of training images for clam_chowder : 1000
Training samples for clam_chowder : ['2989745.jpg', '1501360.jpg', '546100.jpg', '655847.jpg', '21129.jpg'] 

Number of training images for paella : 1000
Training samples for paella : ['1662668.jpg', '61029.jpg', '3629987.jpg', '2327235.jpg', '3177465.jpg'] 

Number of training images for peking_duck : 1000
Training samples for peking_duck : ['151713.jpg', '499238.jpg', '7934.jpg', '535775.jpg', '1177445.jpg'] 

Number of training images for cannoli : 1000
Training samples for cannoli : ['2788362.jpg', '7336.jpg', '410548.jpg', '656936.jpg', '1729081.jpg'] 

Number of training images for cheesecake : 1000
Training samples for cheesecake : ['1435236.jpg', '1093939.jpg', '2912548.jpg', '2363100.jpg', '3904984.jpg']
In [14]:
# Plot some training images from the dataset
nrows = len(CLASS_LABELS)
ncols = 4
training_examples = []
example_labels = []

fig = plt.gcf()
fig.set_size_inches(ncols * 4, nrows * 3)

for c_label in CLASS_LABELS:
    training_class_dir = os.path.join(TRAIN_DIR, c_label)
    training_class_files = os.listdir(training_class_dir)
    for j in range(ncols):
        training_examples.append(training_class_dir + '/' + training_class_files[j])
        example_labels.append(c_label)
    # print(training_examples)
    # print(example_labels)

for i, img_path in enumerate(training_examples):
    # Set up subplot; subplot indices start at 1
    sp = plt.subplot(nrows, ncols, i+1)
    sp.text(0, 0, example_labels[i])
    # sp.axis('Off')
    img = mpimg.imread(img_path)
    plt.imshow(img)
plt.show()
In [15]:
df_train_images = pd.read_csv(TRAIN_LABELS, names=['image'])
df_train_images['image'] = df_train_images['image'] + '.jpg'
df_train_images['class'] = df_train_images['image'].apply(lambda x: x[0:x.find('/')])
print(df_train_images.head())

                   image      class
0  apple_pie/1005649.jpg  apple_pie
1  apple_pie/1014775.jpg  apple_pie
2  apple_pie/1026328.jpg  apple_pie
3  apple_pie/1028787.jpg  apple_pie
4  apple_pie/1043283.jpg  apple_pie
In [16]:
df_test_images = pd.read_csv(TEST_LABELS, names=['image'])
df_test_images['image'] = df_test_images['image'] + '.jpg'
df_test_images['class'] = df_test_images['image'].apply(lambda x: x[0:x.find('/')])
print(df_test_images.head())

                   image      class
0  apple_pie/1011328.jpg  apple_pie
1   apple_pie/101251.jpg  apple_pie
2  apple_pie/1034399.jpg  apple_pie
3   apple_pie/103801.jpg  apple_pie
4  apple_pie/1038694.jpg  apple_pie
In [17]:
datagen_kwargs = dict(rescale=1./255)
training_datagen = ImageDataGenerator(**datagen_kwargs)
validation_datagen = ImageDataGenerator(**datagen_kwargs)
dataflow_kwargs = dict(class_mode="categorical")

do_data_augmentation = True
if do_data_augmentation:
    training_datagen = ImageDataGenerator(rotation_range=45,
                                          horizontal_flip=True,
                                          vertical_flip=True,
                                          **datagen_kwargs)

print('Loading and pre-processing the training images...')
training_generator = training_datagen.flow_from_dataframe(dataframe=df_train_images,
                                                          directory='food-101/images/',
                                                          x_col='image',
                                                          y_col='class',
                                                          target_size=TARGET_IMAGE_SIZE,
                                                          batch_size=BATCH_SIZE,
                                                          shuffle=True,
                                                          seed=RNG_SEED,
                                                          **dataflow_kwargs)
print('Number of training image batches per epoch of modeling:', len(training_generator))

print('Loading and pre-processing the validation images...')
validation_generator = validation_datagen.flow_from_dataframe(dataframe=df_test_images,
                                                              directory='food-101/images/',
                                                              x_col='image',
                                                              y_col='class',
                                                              target_size=TARGET_IMAGE_SIZE,
                                                              batch_size=BATCH_SIZE,
                                                              shuffle=False,
                                                              **dataflow_kwargs)
print('Number of validation image batches per epoch of modeling:', len(validation_generator))

Loading and pre-processing the training images...
Found 75750 validated image filenames belonging to 101 classes.
Number of training image batches per epoch of modeling: 2368
Loading and pre-processing the validation images...
Found 25250 validated image filenames belonging to 101 classes.
Number of validation image batches per epoch of modeling: 790
In [18]:
if NOTIFY_STATUS: status_notify('(TensorFlow Multi-Class) Task 2 - Load and Prepare Images completed on ' + datetime.now().strftime('%A %B %d, %Y %I:%M:%S %p'))

Task 3 - Define and Train Models¶

In [19]:
if NOTIFY_STATUS: status_notify('(TensorFlow Multi-Class) Task 3 - Define and Train Models has begun on ' + datetime.now().strftime('%A %B %d, %Y %I:%M:%S %p'))
In [20]:
# Define the function for plotting training results for comparison
def plot_metrics(history):
    fig, axs = plt.subplots(1, 2, figsize=(24, 15))
    metrics =  [TRAIN_LOSS, TRAIN_METRIC]
    for n, metric in enumerate(metrics):
        name = metric.replace("_"," ").capitalize()
        plt.subplot(2,2,n+1)
        plt.plot(history.epoch, history.history[metric], color='blue', label='Train')
        plt.plot(history.epoch, history.history['val_'+metric], color='red', linestyle="--", label='Val')
        plt.xlabel('Epoch')
        plt.ylabel(name)
        if metric == TRAIN_LOSS:
            plt.ylim([0, plt.ylim()[1]])
        else:
            plt.ylim([0, 1])
        plt.legend()
In [21]:
# Define the baseline model for benchmarking
def create_nn_model(input_param=INPUT_IMAGE_SHAPE, output_param=NUM_CLASSES, dense_nodes=2048,
                    classifier_activation=CLASSIFIER_ACTIVATION, loss_param=DEFAULT_LOSS,
                    opt_param=DEFAULT_OPTIMIZER, metrics_param=DEFAULT_METRICS):
    base_model = keras.applications.resnet_v2.ResNet50V2(include_top=False, weights='imagenet', input_shape=input_param)
    nn_model = keras.models.Sequential()
    nn_model.add(base_model)
    nn_model.add(keras.layers.Flatten())
    nn_model.add(keras.layers.Dense(dense_nodes, activation='relu')),
    nn_model.add(keras.layers.Dense(output_param, activation=classifier_activation))
    nn_model.compile(loss=loss_param, optimizer=opt_param, metrics=metrics_param)
    return nn_model
In [22]:
# Initialize the neural network model and get the training results for plotting graph
start_time_module = datetime.now()
tf.keras.utils.set_random_seed(RNG_SEED)
baseline_model = create_nn_model()
baseline_model_history = baseline_model.fit(training_generator,
                                            epochs=MAX_EPOCHS,
                                            validation_data=validation_generator,
                                            verbose=1)
print('Total time for model fitting:', (datetime.now() - start_time_module))

Epoch 1/10
2368/2368 [==============================] - 1335s 557ms/step - loss: 2.4455 - accuracy: 0.4118 - val_loss: 1.8601 - val_accuracy: 0.5275
Epoch 2/10
2368/2368 [==============================] - 1296s 547ms/step - loss: 1.6967 - accuracy: 0.5674 - val_loss: 1.8078 - val_accuracy: 0.5364
Epoch 3/10
2368/2368 [==============================] - 1295s 547ms/step - loss: 1.4348 - accuracy: 0.6247 - val_loss: 1.6921 - val_accuracy: 0.5670
Epoch 4/10
2368/2368 [==============================] - 1294s 546ms/step - loss: 1.2655 - accuracy: 0.6639 - val_loss: 1.3693 - val_accuracy: 0.6396
Epoch 5/10
2368/2368 [==============================] - 1298s 548ms/step - loss: 1.1271 - accuracy: 0.6973 - val_loss: 1.2892 - val_accuracy: 0.6621
Epoch 6/10
2368/2368 [==============================] - 1290s 545ms/step - loss: 1.0090 - accuracy: 0.7245 - val_loss: 1.4785 - val_accuracy: 0.6236
Epoch 7/10
2368/2368 [==============================] - 1266s 535ms/step - loss: 0.9042 - accuracy: 0.7501 - val_loss: 1.3087 - val_accuracy: 0.6689
Epoch 8/10
2368/2368 [==============================] - 1263s 533ms/step - loss: 0.8135 - accuracy: 0.7723 - val_loss: 1.2760 - val_accuracy: 0.6770
Epoch 9/10
2368/2368 [==============================] - 1271s 537ms/step - loss: 0.7284 - accuracy: 0.7951 - val_loss: 1.5184 - val_accuracy: 0.6363
Epoch 10/10
2368/2368 [==============================] - 1273s 537ms/step - loss: 0.6580 - accuracy: 0.8119 - val_loss: 1.6597 - val_accuracy: 0.6124
Total time for model fitting: 3:35:38.370870
In [23]:
baseline_model.summary()

Model: "sequential"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 resnet50v2 (Functional)     (None, 7, 7, 2048)        23564800  
                                                                 
 flatten (Flatten)           (None, 100352)            0         
                                                                 
 dense (Dense)               (None, 2048)              205522944 
                                                                 
 dense_1 (Dense)             (None, 101)               206949    
                                                                 
=================================================================
Total params: 229,294,693
Trainable params: 229,249,253
Non-trainable params: 45,440
_________________________________________________________________
In [24]:
plot_metrics(baseline_model_history)
In [25]:
if NOTIFY_STATUS: status_notify('(TensorFlow Multi-Class) Task 3 - Define and Train Models completed on ' + datetime.now().strftime('%A %B %d, %Y %I:%M:%S %p'))

Task 4 - Tune and Optimize Models¶

In [26]:
if NOTIFY_STATUS: status_notify('(TensorFlow Multi-Class) Task 4 - Tune and Optimize Models has begun on ' + datetime.now().strftime('%A %B %d, %Y %I:%M:%S %p'))
In [27]:
# Not applicable for this iteration of modeling
In [28]:
if NOTIFY_STATUS: status_notify('(TensorFlow Multi-Class) Task 4 - Tune and Optimize Models completed on ' + datetime.now().strftime('%A %B %d, %Y %I:%M:%S %p'))

Task 5 - Finalize Model and Make Predictions¶

In [29]:
if NOTIFY_STATUS: status_notify('(TensorFlow Multi-Class) Task 5 - Finalize Model and Make Predictions has begun on ' + datetime.now().strftime('%A %B %d, %Y %I:%M:%S %p'))
In [30]:
# Not applicable for this iteration of modeling
In [31]:
if NOTIFY_STATUS: status_notify('(TensorFlow Multi-Class) Task 5 - Finalize Model and Make Predictions completed on ' + datetime.now().strftime('%A %B %d, %Y %I:%M:%S %p'))
In [32]:
print ('Total time for the script:',(datetime.now() - START_TIME_SCRIPT))

Total time for the script: 3:37:44.604873